Skip to content

python(feat): expose include_received_at in get_data - #754

Draft
wei-qlu wants to merge 2 commits into
mainfrom
python/expose-ingest-time
Draft

python(feat): expose include_received_at in get_data#754
wei-qlu wants to merge 2 commits into
mainfrom
python/expose-ingest-time

Conversation

@wei-qlu

@wei-qlu wei-qlu commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Summary

Exposes GetDataRequest.include_received_at field

  • client.channels.get_data(...), get_data_as_arrow(...), and Channel.data(...) now take include_received_at
  • When set, each returned DataFrame gains a "<channel_name>.sift_received_at" column with the time each point was received by Sift, NaT for points without one
  • Not available for enum or bitfield channels (their protos carry no extras field)
  • Received-at data is never cached, so these calls always fetch from the server

Hardening for the new nullable column in _merge_pages:

  • Row-level dedup on duplicate timestamps instead of per-column groupby.last(), which skips nulls and could pair a fresh value with a stale row's received-at
  • Explicit datetime64[ns, UTC] dtype so an all-NaT page doesn't degrade the merged column to object dtype

Test plan

  • Extras parsing: prefixed column, NaT for unset points, foreign extras ignored, length-mismatch guard, real bitfield/enum payloads unaffected
  • Request flag set vs unset on the proto, cache bypass (read and write)
  • Page merge with duplicate timestamps and all-NaT pages
  • Flag forwarding pinned at each layer: Channel.data, ChannelsAPI.get_data, low-level client
image

@github-actions

github-actions Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Python docs preview: https://sift-stack.github.io/sift/python/pr-754/

Deployed from f44c5bb. The link may take up to a minute to become live as GitHub Pages propagates.

@wei-qlu
wei-qlu force-pushed the python/expose-ingest-time branch from fdb91f4 to 9b4f340 Compare August 26, 2026 20:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant